In the United States Army, tele-operation of Unmanned Ground Vehicles (UGVs) is widely employed for the control of small UGVs (S-UGV) performing missions such as Counter Improvised Explosive Device (C-IED) and Explosive Ordinance Disposal (EOD). For these missions, the speeds are generally low and the operator is usually in close proximity to the S-UGV, yielding low latency due to wireless communication and, therefore, responsive control is easily achieved wherein the human operator functions as they would in an actual vehicle.
Tele-operation over relatively long-distances, however, may (depending upon the speed and quality of the wireless communications links available, among other factors) introduce significant latency which can degrade the operator's ability to drive/control the vehicle. Delays in closed-loop control systems are one of the well-known sources of degraded performance and/or stability. Although, in the case of tele-operation, the human-in-the-loop is a stabilizing factor and may provide robust compensation against instability, the human's ability to respond to the visual inputs begins to degrade at delays above 50 ms and performance is so degraded at 200-300 ms that the operator must change his/her control approach from continuous steer to the slower and more error prone “move and wait.” The “move and wait” approach requires the operator reduce speed to mitigate against the delay, which significantly degrades performance and lowers the achievable top speed. As such, the mitigation of this latency is a fundamental challenge to achieving tele-operation under high latency.
There also exists the problem of predicting system behavior in the presence of delays and the challenge of incorporating the prediction into the control algorithm. Some methods strive to place the predictors in-line either predicting a future state of the system or of the operator. Since the standard tele-operation scenario incorporates a video feed being sent back to the operator, this is still delayed by the amount of transport delay between the vehicle and the operator. Furthermore, the video's data, being a series of raster images, is not subject to explicit prediction because its values are not the result of a natural evolution of system states. In this case, researchers have undertaken methods to present information in the video stream to help the operator understand the true state of the system. In these cases, many have undertaken to overlay the display with graphics to include vehicle surrogates and lane markers. Others have undertaken to physically manipulate the video images to estimate what the driver would see if the stream were not delayed.